Conversion Between Aspect Ratios in Camera

ABSTRACT

A camera system captures an image in a source aspect ratio and applies a transformation to the input image to scale and warp the image to generate an output image having a target aspect ratio different than the source aspect ratio. The output image has the same field of view as the input image, maintains image resolution, and limits distortion to levels that do not substantially affect the viewing experience. In one embodiment, the output image is non-linearly warped relative to the input image such that a distortion in the output image relative to the input image is greater in a corner region of the output image than a center region of the output image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/536,315 entitled “Conversion Between Aspect Ratios in Camera” toNicholas D. Woodman, et al., filed on Nov. 7, 2014, which is acontinuation of U.S. patent application Ser. No. 14/180,887 entitled“Conversion Between Aspect Ratios in Camera” to Nicholas D. Woodman, etal., filed on Feb. 14, 2014, which claims the benefit of U.S.Provisional Patent Application No. 61/869,029 entitled “ConversionBetween Aspect Ratios in Camera,” to Nicholas D. Woodman, et. al., filedon Aug. 22, 2013, the contents of which are each incorporated byreference herein.

BACKGROUND

Converting between aspect ratios is a common problem in the field ofimage and video processing. For example, a display that displays imagesaccording to a 16:9 aspect ratio may receive images natively capturedaccording to a 4:3 aspect ratio and perform an aspect ratio conversionto fit the image to the display. Conventional conversion techniques suchas cropping, linearly scaling, and padding each result in a perceivablereduction in the quality of the image or video. For example, croppingremoves content from the image and reduces the field of view of theimage. Linear scaling maintains the full field of view but introducesperceivable distortion into the image. For example, if an image isvertically compressed and/or horizontally stretched to convert from a4:3 to a 16:9 aspect ratio, features in scene will appear “short andfat” relative to the original image. Padding maintains the full field ofview without distortion, but results in unused display space around thesides and/or top of the image, thus rendering an image that appears toosmall relative to the display.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an embodiment of a camera system.

FIG. 2A is a flowchart illustrating an embodiment of a process forconverting an image from a source aspect ratio to a target aspect ratio.

FIG. 2B is set of example images being converted from a source aspectratio to a target aspect ratio.

FIG. 3 is a first set of example images showing an original image in asource aspect ratio, a cropped image converted to a target aspect ratiousing a cropping technique, and a non-linear warped image converted tothe target aspect ratio using a non-linear warping technique.

FIG. 4 is a second set of example images showing an original image in asource aspect ratio, a cropped image converted to a target aspect ratiousing a cropping technique, and a non-linear warped image converted tothe target aspect ratio using a non-linear warping technique.

FIG. 5 is a graph illustrating an embodiment of a function for applyinga horizontal warp to an image.

FIG. 6 is a graph illustrating an embodiment of a function for applyinga vertical warp to an image.

DETAILED DESCRIPTION

The Figures (FIGS.) and the following description relate to preferredembodiments by way of illustration only. It should be noted that fromthe following discussion, alternative embodiments of the structures andmethods disclosed herein will be readily recognized as viablealternatives that may be employed without departing from the principlesof what is claimed.

Reference will now be made in detail to several embodiments, examples ofwhich are illustrated in the accompanying figures. It is noted thatwherever practicable similar or like reference numbers may be used inthe figures and may indicate similar or like functionality. The figuresdepict embodiments of the disclosed system (or method) for purposes ofillustration only. One skilled in the art will readily recognize fromthe following description that alternative embodiments of the structuresand methods illustrated herein may be employed without departing fromthe principles described herein.

Overview

A camera system comprises an image sensor, a processor, and anon-transitory computer-readable storage medium that stores instructionsexecutable by the processor. The image sensor captures an input imagehaving a source aspect ratio. The processor applies a transformation tothe input image to scale and warp the image to generate an output imagehaving a target aspect ratio different than the source aspect ratio. Theoutput image has the same field of view as the input image, maintainsimage resolution, and limits distortion to levels that do notsubstantially affect the viewing experience. In one embodiment, theoutput image is non-linearly warped relative to the input image suchthat a distortion in the output image relative to the input image isgreater in a corner region of the output image than a center region ofthe output image. The processor outputs the output image having thetarget aspect ratio.

In one embodiment, the aspect ratio conversion is performed directly onthe camera itself rather than in an external post-processing device,such as the display device. This enables the camera to output imagesdirectly to a display in the desired target aspect ratio withoutrequiring conversion by the display device. For example, a camera thatnatively captures images according to a 4:3 aspect ratio can performconversion and output the images according to a 16:9 aspect ratio thatmay be displayed directly on a 16:9 display device. Furthermore, in oneembodiment, the conversion may occur in substantially real-time suchthat images captured in the source aspect ratio can be outputted in thetarget aspect ratio with substantially no delay.

Beneficially, since the aspect ratio conversion preserves the entirefield of view of the image, the converted image can be reverted back toits original aspect ratio without any loss of field of view orresolution. Furthermore, since the aspect ratio discrepancy is handledby the camera system, the end user needs not to make any additionaladjustments apart from selecting the desired output video format on thecamera. The embodiments described herein may apply to both individualimages and video (which comprises a sequence of images (or “frames”).

Example Camera System

FIG. 1 illustrates an embodiment of an example camera system 102. Thecamera system 102 comprises an image sensor 104, video storage 106, anda processing unit 108. Other optional components of the camera system102 such as control interfaces, display screen, etc. are omitted fromthe figure for clarity of description. The image sensor 104 capturesdigital images or video. Because video comprises a sequence of images(or “frames”), description herein referring to images can be similarlyapplied to video and vice versa. Each captured image comprises atwo-dimensional array of pixels. The captured frames depict a “scene,”which may include, for example, landscape, people, objects, etcrepresented by the captured pixels. Each pixel represents a depictedpoint in a scene captured in the digital video. Furthermore, each pixelis located at a pixel location, referring to, for example, (x,y)coordinates of the pixel within the frame. For example, a pixel maycomprise {Red, Green, Blue} (RGB) values describing the relativeintensities of the colors sensed by the image sensor 104 at a particularset of (x,y) coordinates in the frame. In various embodiments, the imagesensor 104 may capture video suitable for providing output videos havinghigh definition resolutions (for example, 4 k resolution, 2 kresolution, 1080p, 1080i, 960p, or 720p), standard definitionresolutions, or other resolutions. The image sensor 104 may capturevideo at various frame rates such as, for example, 120 frames perseconds (FPS), 60 fps, 48, fps, or 30 fps, although other frame ratesmay also be used. Additionally, the image sensor 104 may include a lensthat allows for wide-angle or ultra wide-angle video capture havingfields of view of, for example, 90 degrees, 127 degrees, or 170 degrees,although other field of view angles may also be used.

Image frames captured by the image sensor 102 may be temporarily orpermanently stored in the video storage 106 which may comprise, forexample, a hard drive or a solid-state memory device (e.g., a memorycard or universal serial bus (USB) memory device). The processing unit108 processes images to convert between aspect ratios as will bedescribed in further detail below. In one embodiment, the processingunit 108 comprises a processor 112 and a memory 114. The memory 114comprises a non-transitory computer-readable storage medium that storescomputer-executable program instructions for aspect ratio conversion,among other functions. To perform the aspect ratio conversion describedherein, the processor 112 loads program instructions from the memory 114and executes the instructions. Alternatively, the processing unit 108could implement the aspect ratio conversion in hardware, firmware, or acombination of hardware, firmware, and/or software.

In one embodiment, the processing unit 108 converts between a sourceaspect ratio (i.e., the aspect ratio of the original images captured bythe image sensor 104) and a target aspect ratio (i.e., the aspect ratioof the image for storing or outputting by the camera system 102). In oneembodiment, the source aspect ratio makes full use of the availablefield of view of the image sensor 104 (remaining sensitive to mappingdistortion effects). For example, in a camera system 102 having a 4:3image sensor, the image sensor 104 captures the native content accordingto a 4:3 aspect ratio, although other source aspect ratios are alsopossible. The processing unit 108 then uses resolution compressionand/or stretching in the appropriate axes to convert the captured imagesin the source aspect ratio to the target aspect ratio. In oneembodiment, the conversion is achieved using a resolution remapping thatapplies a non-linear function across the image so as to minimize orreduce undesirable key feature aspect ratio distortions. Furthermore, inone embodiment, the aspect ratio conversion does not result in any lossof field of view.

In one embodiment, the aspect ratio conversion is performed by a warpmodule of the processing unit 108. In addition to performing the aspectratio conversion, the warp module may perform other functions such as,for example, reducing optical lens distortions such as typical barreland pincushion distortions.

Aspect Ratio Conversion

Examples of techniques for aspect ratio conversion are discussed infurther detail below. For the sake of clarity, the following descriptiondiscusses examples involving conversions between a source aspect ratioof 4:3 and a target aspect ratio of 16:9. However, the describedembodiments similarly apply to any source or target aspect ratios.

FIG. 2A illustrates an example embodiment of a process for convertingfrom a source aspect ratio to a target aspect ratio in a camera system.In the process of FIG. 2A, an input image is obtained 202 in the sourceaspect ratio (e.g., a 4:3 aspect ratio). The image is then scaled 204from the source aspect ratio (e.g., the 4:3 aspect ratio) to the targetaspect ratio (e.g., a 16:9 aspect ratio) using a linear scalingfunction. A non-linear warping is then applied 206 to the scaled image.FIG. 2B illustrates the process of FIG. 2A as applied to an exampleimage 212 containing an array of circles. As can be seen, the scalingfunction applied in step 204 causes the circles to be stretchedhorizontally and compressed vertically as shown in example image 214.Because the scaling function applied in step 204 is linear, each circleis stretched/compressed equally independent of horizontal and verticalposition. In the warping step 206, a non-linear warping is applied whichresults in the circles being stretched/compressed by different amountsdepending on their horizontal and vertical positions resulting inexample image 216.

As can also be observed from FIG. 2B, the scaling and warping steps 204,206 preserves the full field of view in the resulting image 216 relativeto the original image 212. To achieve this, the warp module isconfigured so that some areas of the image are fully correctedgeometrically, while some other areas are distorted. In the illustratedexample, the center of the image is chosen to be the region of interestthat is most fully corrected and the distortion increases in eachdirection as the frame boundary is approached. Thus, in image 216, thecenter circle is fully corrected so that it maintains the originalcircular shape of the corresponding circle in the input image 212. Inthe vertical direction, the circles become more distorted (e.g., morevertically compressed) in the as they approach the top and bottom edges.In the horizontal direction, the circles become more distorted (e.g.,more horizontally stretched) as they approach the left and right edges.As a result, the circles in the corners exhibit the most distortion,because they have the highest amount of both vertical compression andhorizontal stretching. In other embodiments, a different part of theimage may be selected as the area most fully corrected, or in the case,of video, the area may dynamically change between images (e.g., tofollow a moving object).

The distribution of distortion levels in different portions of the imagemay furthermore be tuned in one embodiment to control the size ofobjects as they pan from horizontally within a scene of video. Fullycorrecting the geometrical distortion in the center of the image wouldresult in objects looking small in the center and very large on thesides. Thus, in the case of a video, an object may noticeably grow insize as it moves from the center to the edges of the display. In oneembodiment, the warping function in step 206 is tuned to reduce thiseffect.

In an alternative embodiment, the scaling and warping functions appliedin steps 204, 206 could be performed in reverse order, such that thewarping function in step 206 is applied first followed by scaling instep 204. In another embodiment, the scaling and warping are combinedinto a single function such that only a single transformation is appliedto the image that achieves both the scaling and warping. For example, inFIG. 2B, image 212 may be converted directly to image 216 via a singlefunction without producing the intermediate image 214.

As will be apparent to those skilled in the art, an equitable horizontalremapping can be performed in manners similar to those described hereinfor vertical remapping if it is desired to decrease image width withoutdecreasing horizontal image field of view and without creatingundesirable key feature distortions. For example, images having a sourceaspect ratio of 2.39:1 may be converted to a 16:9 aspect ratio in thissimilar manner.

The scaling step (e.g., step 204 of FIG. 2A) may be achieved either byscaling down one dimension (either the horizontal or vertical one,depending on the formats being converted) or scaling up the orthogonaldirection. Scaling up has the benefit of preserving one dimension andonly interpolates existing information to extend the other dimension,meaning all resolution is preserved. On the contrary, scaling downcauses a loss of information.

FIG. 3 illustrates example images illustrating the aspect ratioconversion technique described herein. Image 302 illustrates an originalimage captured with a 4:3 aspect ratio. Image 304 illustrates aconventional cropping of the image to fit it to a 16:9 aspect ratio. Ascan be seen, the field of view is reduced with this technique and aportion of the image is lost. Image 306 illustrates the original imageconverted according to the techniques described herein. As can be seen,the aspect ratio conversion is achieved while maintaining the full fieldof view. Although some distortion may be introduced, the distortion isconcentrated near the boundaries of the image (away from the center) andperceivable distortion is limited to an acceptable level.

FIG. 4 illustrates additional example images representing a checkerboardpattern, where image 502 represent an original 4:3 image, image 504represents an image that has been cropped to achieve the 16:9 aspectratio, and image 506 illustrates an image converted according to thetechniques described herein. As can be seen, the perceivable distortionis small or zero near the center of image and increases somewhat at theedges and corners.

Examples of Warping Functions

In one embodiment, the warping function involves both a horizontal andvertical warping. For convenience of description, the horizontal andvertical warping is described as two steps, e.g., first horizontal thenvertical. However, both horizontal and vertical warping can be executedin one step, for example, by applying a single transformation to theimage. Furthermore, due to the symmetrical nature of the distortion,vertical and horizontal distortion profiles can be swapped.

An example of a horizontal warping function is illustrated in FIG. 5. Inthis graph, the horizontal coordinate of a pixel is expressed as apercentage of the horizontal pixel position relative to the horizontalwidth of the image, where the percentage increases in a direction awayfrom a reference edge. The vertical coordinate is similarly expressed.The amount of distortion is expressed in terms of the horizontaldistortion offset AH that a point in the image moves in a direction awayfrom the reference edge divided by the horizontal image size H. Sincethe warping function is horizontally symmetric in this embodiment, thereference edge can be either the right edge or the left edge. In thisembodiment, the horizontal warp is function of the horizontal coordinateonly and is independent of vertical position. The horizontal warpdistortion stretches pixels horizontally in a manner that corrects forthe “short and fat” effect introduced by 4:3 to 16:9 stretching, orsimilarly undesirable effects introduced by other aspect ratioconversions. In one embodiment of the horizontal warp, pixels alongvertical edges are not offset (the offset value is zero) because thereis no information beyond the edges and it may be undesirable tointroduce padding (e.g., black vertical lines) on the sides of theimage. The magnitude of the distortion is symmetric in one embodiment,with pixels on the left half of the image shifting to the right andpixels on the right half of the image shifting to the left to compensatefor the horizontal stretching effect. Since the center receives thecontributions of pixels from both sides, the center region of the imageis corrected the most, while the geometry on the sides is distorted to asmall extent. This degradation in image quality along the sides can becompensated for to some extent in the vertical warping, as will bediscussed below.

In one embodiment, the amount of horizontal correction applied can bemanually tuned to reduce perceivable distortion in the output image. Forexample, fully correcting the geometrical distortion in the center maycause the objects in the center to be much smaller than on the sides.Thus, in one embodiment, the amount of horizontal correction can betuned in combination with the vertical correction in order to preservethe width of objects throughout the horizontal plane.

In one embodiment, a similarly shaped function is used for verticalwarping. In another embodiment, the vertical warping function ismodified so that the amount of compensation in the vertical directionalso depends on horizontal position. FIG. 6 illustrates an example of avertical warping function, where the amount of distortion is express interms of the vertical distortion offset ΔV indicating an amount that apixel moves in a vertical direction away from a reference edge (e.g.,the top edge or the bottom edge) divided by the vertical image size V.In one embodiment, vertical warping is a function of both the horizontaland vertical coordinates.

As mentioned above, a goal of the vertical distortion is to furthercorrect the geometry that was not already corrected by horizontaldistortion. The image center could be fully corrected using thehorizontal distortion only (with no vertical warping). However,spreading the correction along both horizontal and vertical axiseffectively improves the geometry of not only the center, but also thetop, bottom, left and right image sides. The most distorted regions inthis embodiment are the four image corners, but this distortion does notsubstantially reduce perceived image quality. Having residual distortionin the corners while correcting the rest in the image results in a muchmore natural-looking effect since key scene features are usually notframed in the corners. Examples of this effect can be visible, however,in FIG. 4, where the squares in the corners are more rectangular (asopposed to square), and FIG. 3 where the diver's right flipper issomewhat stretched. However, as a good trade off for these cornerdistortions, it can be observed that the central rows of squares in FIG.4 maintain a relatively consistent width throughout the image.Furthermore, the diver's body in FIG. 3 has proportions that arerelatively preserved across the image width.

In one embodiment, tunable parameters can be controlled to adjust thevertical distortion along both horizontal and vertical directions,resulting in the 3D-wave in FIG. 7. In one embodiment, pixels are notoffset along the horizontal edges to avoid introducing horizontalpadding (e.g., black bands along the edges).

In the description above, the horizontal and vertical distortions havebeen decoupled for the sake of clarity. However, both distortions couldbe implemented in a single step. Similarly, in the description above,the aspect ratio conversion (4:3 to 16:9 in this example) has beendecoupled from warp distortion, but these could also be combined into asingle step.

Example equations are provided below for implementing an example warpingfunction applied in step 206 of FIG. 2. Those of ordinary skill in theart will understand that variations of these equations may also be used,and other implementations may apply that result in functions havingsubstantially similar characteristics.

As explained above, the horizontal warp distortion in one embodiment isa function of the horizontal position only. It can therefore berepresented in terms of a horizontal coordinate h in an equation such asthe example equation below, or an equation substantially similar to theequation below:

Δh/H=0.1107h ⁶+0.534h ⁵−1.7813h ⁴+1.3167h ³−0.0722h ²−0.1073h

The vertical warp distortion, in one embodiment, is a function of boththe horizontal and vertical positions. Therefore, the vertical warpdistortion can be represented as a family of equations ΔV/V_(n), where“n” is the horizontal coordinate expressed as a percent of the totalhorizontal image size. Intermediate horizontal coordinates can beinterpolated from the example equations given below, or equationssubstantially similar to these:

Δv/V ₀=−0.0749v ⁶−0.3167v ⁵+1.0939v ⁴−0.8113v ³+0.0408v ²+0.0678v

Δv/V ₁₀=−0.0658v ⁶−0.2785v ⁵+0.9617v ⁴−0.7132v ³+0.0359v ²+0.0596v

Δv/V ₂₀=−0.0568v ⁶−0.2402v ⁵+0.8294v ⁴−0.6152v ³+0.0309v ²+0.0514v

Δv/V ₃₀=−0.0444v ⁶−0.188v ⁵+0.6491v ⁴−0.4814v ³+0.0242v ²+0.0403v

Δv/V ₄₀=−0.0337v ⁶−0.1427v ⁵+0.4928v ⁴−0.3655v ³+0.0184v ²+0.0306v

Δv/V ₅₀=−0.0296v ⁶−0.1253v ⁵+0.4327v ⁴−0.3209v ³+0.0161v ²+0.0268v

Δv/V ₆₀=−0.0337v ⁶−0.1427v ⁵+0.4928v ⁴−0.3655v ³+0.0184v ²+0.0306v

Δv/V ₇₀=−0.0444v ⁶−0.188v ⁵+0.6491v ⁴−0.4814v ³+0.0242v ²+0.0403v

Δv/V ₈₀=−0.0568v ⁶−0.2402v ⁵+0.8294v ⁴−0.6152v ³+0.0309v ²+0.0514v

Δv/V ₉₀=−0.0658v ⁶−0.2785v ⁵+0.9617v ⁴−0.7132v ³+0.0359v ²+0.0596v

Δv/V ₁₀₀=−0.0749v ⁶−0.3167v ⁵+1.0939v ⁴−0.8113v ³+0.0408v ²+0.0678v

The warp distortion described above may be used with cameras producingeither rectilinear images (without distortion) or images affected byoptical lens distortion characterized by the following equation:

(FOV)=(21.904)*(IH)−(0.6852)*(IH)̂2.

where FOV is the field of view and IH is image height. As will beapparent to those of ordinary skill in the art, the equations above mayvary when used with lenses having different distortion characteristics.

As described above, the above equations can be combined with a scalingfunction to generate a transformation that performs both scaling andwarping.

In alternative embodiments, different equations may be used to achievesimilar distortion characteristics. As mentioned above, the symmetricnature of the distortion and the fact that it can be expressed as apercent of image coordinates allows swapping horizontal and verticaldistortions. Thus, in an alternative embodiment, the equations above forthe horizontal axis can instead by applied in the vertical direction andvice versa.

Furthermore, in other alternative embodiments, the equations may changedynamically in different frames of a video. For example, warping may beapplied in a manner that reduces distortion applied to a particularobject in motion across different frames of a video since the viewer islikely to focus attention on the object in motion.

Although the example embodiments described herein relate to conversionbetween a 4:3 aspect ratio and a 16:9 aspect ratio, other aspect ratiosmay be used for the target and source aspect ratios. For example, inalternative embodiments the source aspect ratio can comprise a 4:3aspect ratio, a 16:9 aspect ratio, a 1.85:1 aspect ratio, a 2.39:1aspect ratio, a 3:2 aspect ratio, a 5:3 aspect ratio, a 5:4 aspectratio, a 1:1 aspect ratio or any other aspect ratio. Similarly, inalternative embodiments, the target aspect ratio can comprise a 4:3aspect ratio, a 16:9 aspect ratio, a 1.85:1 aspect ratio, a 2.39:1aspect ratio, a 3:2 aspect ratio, a 5:3 aspect ratio, a 5:4 aspectratio, a 1:1 aspect ratio or any other aspect ratio.

Additional Configuration Considerations

Throughout this specification, some embodiments have used the expression“coupled” along with its derivatives. The term “coupled” as used hereinis not necessarily limited to two or more elements being in directphysical or electrical contact. Rather, the term “coupled” may alsoencompass two or more elements are not in direct contact with eachother, but yet still co-operate or interact with each other, or arestructured to provide a thermal conduction path between the elements.

Likewise, as used herein, the terms “comprises,” “comprising,”“includes,” “including,” “has,” “having” or any other variation thereof,are intended to cover a non-exclusive inclusion. For example, a process,method, article, or apparatus that comprises a list of elements is notnecessarily limited to only those elements but may include otherelements not expressly listed or inherent to such process, method,article, or apparatus.

In addition, use of the “a” or “an” are employed to describe elementsand components of the embodiments herein. This is done merely forconvenience and to give a general sense of the invention. Thisdescription should be read to include one or at least one and thesingular also includes the plural unless it is obvious that it is meantotherwise.

Finally, as used herein any reference to “one embodiment” or “anembodiment” means that a particular element, feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment. The appearances of the phrase “in oneembodiment” in various places in the specification are not necessarilyall referring to the same embodiment.

Upon reading this disclosure, those of skill in the art will appreciatestill additional alternative structural and functional designs for acamera expansion module as disclosed from the principles herein. Thus,while particular embodiments and applications have been illustrated anddescribed, it is to be understood that the disclosed embodiments are notlimited to the precise construction and components disclosed herein.Various modifications, changes and variations, which will be apparent tothose skilled in the art, may be made in the arrangement, operation anddetails of the method and apparatus disclosed herein without departingfrom the spirit and scope defined in the appended claims.

1. A method for converting between aspect ratios, comprising: obtainingan input image having a source aspect ratio; applying, by a processingdevice, a transformation to the input image to scale and warp the imageto generate an output image having a target aspect ratio, the targetaspect ratio different than the source aspect ratio, the output imagenon-linearly warped relative to the input image such that a distortionin the output image relative to the input image is greater in a cornerregion of the output image than a center region of the output image,wherein applying the transformation comprises shifting pixels of theinput image by an amount represented by a non-linear function oforiginal pixel positions of the pixels; outputting the output imagehaving the target aspect ratio.
 2. The method of claim 1, whereinapplying the transformation to the input image to scale and warp theimage comprises: applying a scaling function to the input image tolinearly stretch or compress the input image to generate a scaled imagehaving the target aspect ratio; and applying a non-linear warpingfunction to the scaled image to reduce distortion of a center region ofthe scaled image relative to the input image and increase distortion ofa corner region of the scaled image relative to the input image.
 3. Themethod of claim 1, wherein applying the transformation to the image toscale and warp the image comprises: applying a non-linear warpingfunction to the input image to generate a warped image, the warped imagehaving a reduced distortion of a center region of the warped imagerelative to the input image and an increased distortion of a cornerregion of the warped image relative to the input image; applying ascaling function to the warped image to linearly stretch or compress thewarped image to generate the output image having the target aspectratio.
 4. The method of claim 1, wherein applying the transformationcomprising applying a combined function that performs both warping andscaling.
 5. The method of claim 1, wherein the processing devicecomprises a camera.
 6. The method of claim 1, wherein the output imagehas a same field of view as the input image.
 7. The method of claim 1,wherein the source aspect ratio comprises a 4:3 aspect ratio and thetarget aspect ratio comprises a 16:9 aspect ratio.
 8. A non-transitorycomputer-readable storage medium storing instructions for convertingbetween aspect ratios, the instructions when executed by a processorcausing the processor to perform steps including: obtaining an inputimage having a source aspect ratio; applying a transformation to theinput image to scale and warp the image to generate an output imagehaving a target aspect ratio, the target aspect ratio different than thesource aspect ratio, the output image non-linearly warped relative tothe input image such that a distortion in the output image relative tothe input image is greater in a corner region of the output image than acenter region of the output image, wherein applying the transformationcomprises shifting pixels of the input image by an amount represented bya non-linear function of original pixel positions of the pixels andoutputting the output image having the target aspect ratio.
 9. Thenon-transitory computer-readable storage medium of claim 8, whereinapplying the transformation to the input image to scale and warp theinput image comprises: applying a scaling function to the input image tolinearly stretch or compress the input image to generate a scaled imagehaving the target aspect ratio; and applying a non-linear warpingfunction to the scaled image to reduce distortion of a center region ofthe scaled image relative to the input image and increase distortion ofa corner region of the scaled image relative to the input image.
 10. Thenon-transitory computer-readable storage medium of claim 8, whereinapplying the transformation to the input image to scale and warp theinput image comprises: applying a non-linear warping function to theinput image to generate a warped image, the warped image having areduced distortion of a center region of the warped image relative tothe input image and an increased distortion of a corner region of thewarped image relative to the input image; applying a scaling function tothe warped image to linearly stretch or compress the warped image togenerate the output image having the target aspect ratio.
 11. Thenon-transitory computer-readable storage medium of claim 8, whereinapplying the transformation comprises applying a combined function thatperforms both warping and scaling.
 12. The non-transitorycomputer-readable storage medium of claim 8, wherein the output imagehas a same field of view as the input image.
 13. The non-transitorycomputer-readable storage medium of claim 8, wherein the source aspectratio comprises a 4:3 aspect ratio and the target aspect ratio comprisesa 16:9 aspect ratio.
 14. A computing apparatus comprising: an inputdevice to obtain an input image having a source aspect ratio; aprocessor to process the input image captured by the image sensor; and anon-transitory computer-readable storage medium storing instructionsthat when executed by the processor cause the processor to perform stepsincluding: applying a transformation to the input image to scale andwarp the image to generate an output image having a target aspect ratio,the target aspect ratio different than the source aspect ratio, and theoutput image non-linearly warped relative to the input image such that adistortion in the output image relative to the input image is greater ina corner region of the output image than a center region of the outputimage, wherein applying the transformation comprises: shifting a pixelof the input image along a first axis as a non-linear function of anoriginal pixel position of the pixel along the first axis, independentlyof an original pixel position of the pixel along a second axisperpendicular to the first axis; and shifting the pixel of the inputimage along the second axis as a non-linear function of the originalpixel position of the pixel along the first and second axes; and anoutput device to output the output image having the target aspect ratio.15. A computing apparatus comprising: an input device to obtain an inputimage having a source aspect ratio; a processor to process the inputimage captured by the image sensor; and a non-transitorycomputer-readable storage medium storing instructions that when executedby the processor cause the processor to perform steps including:applying a transformation to the input image to scale and warp the imageto generate an output image having a target aspect ratio, the targetaspect ratio different than the source aspect ratio, the output imagenon-linearly warped relative to the input image such that a distortionin the output image relative to the input image is greater in a cornerregion of the output image than a center region of the output image,wherein applying the transformation comprises shifting pixels of theinput image by an amount represented by a non-linear function oforiginal pixel positions of the pixels; and an output device to outputthe output image having the target aspect ratio.
 16. The computingapparatus of claim 15, wherein applying the transformation to the inputimage to scale and warp the input image comprises: applying a scalingfunction to the input image to linearly stretch or compress the inputimage to generate a scaled image having the target aspect ratio; andapplying a non-linear warping function to the scaled image to reducedistortion of a center region of the scaled image relative to the inputimage and increase distortion of a corner region of the scaled imagerelative to the input image.
 17. The computing apparatus of claim 15,wherein applying the transformation to the input image to scale and warpthe input image comprises: applying a non-linear warping function to theinput image to generate a warped image, the warped image having areduced distortion of a center region of the warped image relative tothe input image and an increased distortion of a corner region of thewarped image relative to the input image; applying a scaling function tothe warped image to linearly stretch or compress the warped image togenerate the output image having the target aspect ratio.
 18. Thecomputing apparatus of claim 15, wherein applying the transformationcomprising applying a combined function that performs both warping andscaling.
 19. The computing apparatus of claim 15, wherein the computingdevice comprises a camera.
 20. The computing apparatus of claim 15,wherein the output image has a same field of view as the input image.